Zillow MMM - Submits
Spencer Smith
Generated on ‘21 January, 2020’
Model Hyper Parameters
Training dates 2016-09-01 to 2019-12-30
Transformations
| Channel | Diminishing Returns Shape | Decay rate |
|---|---|---|
| TV | log | 0.98 |
| Digital | 0.9 | 0.3 |
| SEM Brand | 0.2 | 0.4 |
| SEM Non-Brand | 0.9 | 0.4 |
| Social | 0.9 | 0.6 |
| App | 0.9 | 0.93 |
Model Performance
\(R^2\) = 0.8907251
MAPE = 4.19%
MAPE (Test) = 4.40%
RMSE = 2,389
RMSE (Test) = 2,430
| Estimate | t value | Pr(>|t|) | |
|---|---|---|---|
| (Intercept) | -25233.68 * | -1.441 | 0.150 |
| tv | 9.654 | 3.168 | 0.002 |
| semBR | 155.388 | 2.401 | 0.017 |
| semNB | 0.043 | 2.884 | 0.004 |
| app | 0.024 | 5.076 | 0.000 |
| social | 0.056 | 3.943 | 0.000 |
| NA | NA | NA | NA |
| email_marketing | 0 * | -0.901 | 0.368 |
| email_operational | 0 | 4.546 | 0.000 |
| email_savedsearch | 0 * | -0.249 | 0.803 |
| SEO_bigcity | 39028.293 | 2.156 | 0.031 |
| SEO_hdp | -14710.895 | -2.533 | 0.011 |
| push_android | 0 * | -1.528 | 0.127 |
| push_ios | 0 | 2.016 | 0.044 |
| nonbrand_queries | 0.3 | 15.498 | 0.000 |
*Not significant effect
Lift Analysis
Lift is the expected contribution of the channel to the overall prediction. “If we removed the channel, what percent of total Submits would we lose?”| 2017 | 2018 | 2019 | |
|---|---|---|---|
| (Intercept) | -55% | -57% | -57% |
| tv | 12% | 11% | 11% |
| semBR | 3% | 4% | 4% |
| semNB | 2% | 3% | 3% |
| app | 4% | 5% | 8% |
| social | 3% | 4% | 6% |
| digital | NA% | NA% | NA% |
| email_marketing | 0% | 0% | 0% |
| email_operational | 5% | 3% | 4% |
| email_savedsearch | 0% | 0% | 0% |
| SEO_bigcity | 85% | 88% | 86% |
| SEO_hdp | -29% | -29% | -28% |
| push_android | -1% | -1% | -1% |
| push_ios | 1% | 1% | 2% |
| nonbrand_queries | 72% | 73% | 77% |
CPA Analysis
| date | tv | semBR | semNB | app | social | digital |
|---|---|---|---|---|---|---|
| 2016-09-01 | $15.41 | $5.03 | $41.39 | $7.25 | $18.54 | Inf |
| 2017-01-01 | $22.46 | $5.38 | $40.58 | $7.59 | $20.03 | Inf |
| 2018-01-01 | $25.16 | $5.81 | $41.16 | $7.99 | $20.59 | Inf |
| 2019-01-01 | $20.93 | $6.42 | $41.07 | $8.06 | $21.20 | Inf |
Error Plot
Variable Analysis
Variable Correlation
Media Marginal Plots
Variable plots
Owned
## [1] "email_marketing" "email_operational" "email_savedsearch"
## [4] "push_android" "push_ios"
Search
## [1] "SEO_bigcity" "SEO_hdp" "nonbrand_queries"
Apendix
Seasonal Variable
| Estimate | t value | Pr(>|t|) | |
|---|---|---|---|
| christmas | -9,569 | -9.842 | 0.000 |
| easter | -4,688 | -4.503 | 0.000 |
| fathers | -2,657 | -2.542 | 0.011 |
| halloween | -2,054 * | -1.614 | 0.107 |
| independence | -9,477 | -6.384 | 0.000 |
| labor | -2,591 | -3.439 | 0.001 |
| martin | 1,130 * | 0.781 | 0.435 |
| memorial | -3,824 | -4.306 | 0.000 |
| mothers | -3,278 | -3.124 | 0.002 |
| new | -6,527 | -6.267 | 0.000 |
| presidents | 2,924 | 2.016 | 0.044 |
| super | -801 * | -0.551 | 0.582 |
| thanksgiving | -4,569 | -6.381 | 0.000 |
| valentines | -4,661 | -3.238 | 0.001 |
| year2017 | 280 * | 0.658 | 0.511 |
| year2018 | -1,459 | -2.390 | 0.017 |
| year2019 | -5,967 | -6.930 | 0.000 |
| month2 | 2,671 | 6.449 | 0.000 |
| month3 | 1,744 | 4.221 | 0.000 |
| month4 | 1,310 | 2.818 | 0.005 |
| month5 | -1,712 | -3.602 | 0.000 |
| month6 | -1,779 | -3.523 | 0.000 |
| month7 | -203 * | -0.394 | 0.694 |
| month8 | -638 * | -1.431 | 0.153 |
| month9 | -2,640 | -6.608 | 0.000 |
| month10 | -3,495 | -8.047 | 0.000 |
| month11 | -4,969 | -9.611 | 0.000 |
| month12 | -7,271 | -15.097 | 0.000 |
| wdayMonday | 3,184 | 7.809 | 0.000 |
| wdaySaturday | -468 * | -1.326 | 0.185 |
| wdaySunday | -2,333 | -5.165 | 0.000 |
| wdayThursday | 1,487 | 4.253 | 0.000 |
| wdayTuesday | 2,526 | 6.633 | 0.000 |
| wdayWednesday | 2,354 | 6.491 | 0.000 |